Towards preferential selection in the prisoner’s dilemma game

In previous works, the choice of learning neighbor for an individual has generally obeyed pure random selection or preferential selection rules. In this paper, we introduce a tunable parameter ε to characterize the strength of preferential selection and focus on the transition towards preferential selection in the spatial evolutionary game by controlling ε to guide the system from pure random selection to preferential selection. Our simulation results reveal that the introduction of preferential selection can hugely alleviate social dilemmas and enhance network reciprocity. A larger ε leads to a higher critical threshold of the temptation b for the extinction of cooperators. Moreover, we provide some intuitive explanations for the above results from the perspective of strategy transition and cooperative clusters. Finally, we examine the robustness of the results for noise K and different topologies, find that qualitative features of the results are unchanged.


Introduction
Cooperation is ubiquitous in nature and human society [1]. However, this is inconsistent with Darwin's theory of natural selection, because selfish individuals often give up cooperation in pursuit of higher payoffs. Therefore, the evolution and maintenance of cooperation is an interesting and attractive topic that has received widespread attention. Evolutionary game theory provides a powerful framework for understanding and unraveling the essence of the emergence of cooperation among unrelated individuals [2]. As a classical paradigm in game theory, the prisoner's dilemma game (PDG) has been widely studied [3,4]. The PDG describes the interaction between two individuals, with both individuals being allowed to cooperate with or betray their opponent. The individual's payoff depends on the strategy that they choose. Mutual cooperation is rewarded R, and mutual defection is punished P. When a defector meets a cooperator, the defector will face temptation T, and the cooperator receives S (the sucker's payoff). These payoffs need to satisfy T > R > P > S and T + S < 2R. Compared with cooperation, the defection is the optimal option. But they will obtain the maximal payoff when both players choose to cooperate. Therefore, a social dilemma is formed.
In the past few decades, researchers have put a great deal of effort into understanding how cooperative behavior can evolve in social dilemmas [5][6][7]. In a pioneering work, Nowak and  May [8] first introduced a spatial structure into the PDG and found that cooperators can form clusters to resist the invasion of defectors. In addition, many specific mechanisms have been proposed to solve the conundrum of the emergence of cooperation, such as reward and punishment [9][10][11][12][13][14][15], persistence [16][17][18], social exclusion [19], reputation [20][21][22][23][24], voluntary participation [25], mobility [26][27][28][29], timescale diversity [30,31], and conformism [32][33][34][35]. In many previous works, individuals were assumed to randomly select their learning neighbors. However, preferential selection is more realistic, so the effects of preferential selection on the evolution of cooperation have received much attention. Wu et al. [36] introduced a dynamic preferential selection mechanism into the evolutionary PDG, and proved that a simple selection rule can promote cooperation. Guan et al. then introduced a nonlinear attraction effect related to the payoff into the evolutionary PDG and considered two kinds of network structures: square lattices [37] and the regular small-world network [38]. Yang et al. [39] studied a degree-related preferential selection mechanism in evolutionary games. Shi et al. [40] found that payoff-based preferential selection can significantly enhance the emergence of public cooperation. Wang and Perc [41,42] found that increasing the probability of selecting the most suitable neighbor can enhance cooperation. Wang et al. [43] proved that age-related preference selection favors the emergence of cooperation. Huang et al. [44] showed that individuals preferring to select neighbors who have a large degree of difference with themselves as a reference can significantly promote cooperation. Wu et al. [45] studied the influence of environment-based preferential selection on cooperation in evolutionary PDG, and showed that it can accelerate the formation of clusters to resist the invasion of defection. More recently, Wang et al. [46] introduced a preferential selection mechanism considering the satisfaction of payoffs and memory in the spatial PDG. Li et al. [47] showed that preferential selection based on dynamic reputation changes can facilitate cooperation in spatial multi-games.
So far, it has been assumed that individuals select role models using pure random selection or preferential selection, but the combined effects of the two selection mechanisms on cooperation have not been fully studied. In real life, people chooses the role model may neither completely according to a certain preferential selection rule nor according to a randomly selection rule. It is possible that some random factors are incorporated into the preferential selection process. In this paper, we study the transition towards preferential selection in the spatial PDG by introducing a normalized parameter ε to guide the system from pure random selection to preferential selection. We show that increasing ε (i.e., during this transition) has an effect on continuously enhancing the emergence and maintenance of cooperation.
The rest of this paper is organized as follows. We first specify the details of the evolutionary game model. We then report and analyze the simulation results. Finally, conclusions are provided in the last section.

Model
In this paper, we consider the weak prisoner's dilemma game on square lattices with periodic boundary conditions and complex networks. The population size is N = 10000 (for the square lattice, L = 100). Each individual occupies a node on the network and interacts with their direct neighbors. Individuals are initially designated either as a cooperator (C) or defector (D). Then, each individual i acquires the accumulated payoff P i through the pairwise interactions with their direct neighbors via the following rules: the reward for mutual cooperation is R = 1, the punishment for mutual defection is P = 0, and when two competing individuals have different strategies, the temptation to defect is T = b (1 � b � 2), and the payoff for the sucker is S = 0. Afterward, all the individuals select a nearest neighbor as the role model for strategy imitation. Specifically, individual i will select one direct neighbor j as a reference according to the probability p ij , where O i is the set of all direct neighbors of the individual i, and k i is the degree of individual i.
Here we introduce an adjustable parameter ε to characterize the strength of preferential selection. Obviously, when ε = 0, this selection scheme returns to the traditional version (random selection). For 0 < ε < 1, the combined effect of random selection and preferential selection is considered. ε = 1 corresponds to the situation in which the role model is selected purely according to the preferential selection rule.
After selecting the role model, all individuals update their strategies simultaneously according to the Fermi rule [48], i.e., the individual i adopts the strategy of the role model j with a probability determined by the Fermi function where K quantifies the amplitude of noise during the strategy imitation, with K = 0.1. Our simulations are performed by iterating the above procedure through over 10 4 time steps, which is sufficient to ensure the system reaches a dynamical equilibrium state. To measure the impact of the combined effect of random selection and preferential selection on the evolution of cooperation, we monitor the cooperation level F C and the data presented below are averaged over up to 50 realizations.

Results
We first consider the effects of the strength of preferential selection ε and the temptation to defect b on the cooperation in evolutionary weak PDG on square lattices, as shown in Fig 1. For ε = 0, the system returns to the traditional version, where the role model of each individual is chosen randomly from the neighbors, and the cooperation level sharply decreases to 0 with increasing temptation. For ε > 0, the preferential selection mechanism is introduced into the system. As ε increases, the individual becomes more likely to select a neighbor with a higher payoff to imitate. One can see that the introduction of the preferential selection mechanism can enhance the evolution of cooperation. To provide a holistic profile of the effects of ε and b on the evolution of cooperation, Fig 1(b) shows a contour plot encoding F c on the ε − b parameter plane. As shown in this contour plot, the evolution of cooperation is influenced by the joint effects of ε and b. When b is fixed, F c rises as we increase the strength of preferential selection ε. Moreover, a larger ε will lead to the need for a higher threshold of temptation b to eliminate cooperative behaviors. Obviously, this indicates that the preferential selection mechanism can tremendously benefit the emergence of cooperation in a population.
To investigate a potential mechanism for the enhancement of the emergence and maintenance of cooperation, we record in Fig 2(a) the time evolution of the fraction of cooperators F c in the weak PDG on square lattices for six different strengths of preferential selection ε (ε = 0.0, 0.2, 0.4, 0.6, 0.8, 1.0). The temptation to defect b = 1.02. From Fig 2(a), one can see that the evolutionary process of the system is divided into two stages, the enduring (END) period and the expanding (EXP) period [49,50]. In the END period, i.e., the early stage of the evolutionary process, F C monotonically decreases no matter the value of ε, since the performance of the defectors is better than that of the cooperators. Moreover, one can observe that a larger ε results in a higher F C and a shorter period of END. In the subsequent EXP period, the time series of F C with different values of ε show two different trends. For ε > 0, F C monotonically increases before reaching a plateau. However, for ε = 0 (the random selection version), F C continues to decrease until reaching a dynamical equilibrium state.

PLOS ONE
Furthermore, we depict the characteristic snapshots of the distribution of different individuals in Fig 2(b)-2(d) to explore how the preferential selection mechanism influences the evolution of cooperation. Initially, all individuals are randomly distributed on the square lattice. In the early stage of the game (i.e., the END period), the snapshots corresponding to different ε do not show much difference. Many boundary and isolated cooperators are changed into defectors as they are exploited by the surrounding defectors (see the second column snapshots). However, the difference between snapshots with ε = 0 and ε > 0 becomes remarkable in the following EXP stage. Compared with the traditional model (ε = 0), the introduction of preferential selection (ε > 0) can significantly boost the expansion of cooperative clusters. For ε > 0, the neighbor with the highest payoff is more likely to be chosen as the role model of a boundary individual. At the beginning of the EXP stage, some small cooperative clusters have been formed, in which the internal cooperators have advantages in payoffs. The boundary

PLOS ONE
individuals are then more inclined to select internal cooperators to imitate. As shown in Fig  2(b)-2(d), the larger ε, the more large and compact the cooperative clusters.
To further support the above explanation, we present the time evolution of the fraction of individuals whose strategies transform from cooperation to defection (defection to cooperation) P C!D (P D!C ) for different ε in Fig 3(a). At the END stage, one can obviously see that P C ! D is always higher than P D!C for any given value of ε, so the cooperators cannot defend against the invasion of the defectors. While it is the reverse for ε = 0.5 and 1.0 when the system is entering into the EXP period, the introduction of preferential selection can facilitate the spreading of cooperation. Moreover, the effect of the strength of preferential selection on the evolution of cooperation is further investigated. We introduce the quantity χ = ρ CC /ρ CL to quantify the compactness of cooperator clusters, where ρ CC is the fraction of cooperator-cooperator links (i.e., links connect two cooperators), and ρ CL denotes the ratio of direct links ending at a cooperator. Clearly, a larger χ means the cooperative clusters in the population are more compact, and χ = 1 means the full cooperation state has been reached. Fig 3(b) shows the equilibrium F C (solid line) and quantity χ (dashed line) as functions of ε for b = 1.02, 1.10, 1.20. For each given b, the equilibrium F C and χ increase monotonically increasing ε, which indicates that the introduction of preferential selection can facilitate and stabilize the formation of compact cooperative clusters, and the cooperators can more easily avoid the invasion of defectors.
To better determine why the preferential selection mechanism can more effectively stabilize the formation of cooperative clusters than pure random selection, we depict three representative spatial evolutionary snapshots on square lattices in Fig 4, in which the simulation is started from a prepared initial condition. In the absence of preferential selection (ε = 0.0), the initial compact cooperative clusters are not able to defend against the invasion of defection, and eventually, the cooperative behavior in the population goes extinct. However, the cooperative clusters are strong when the preferential selection scheme is introduced, and the introduction of preferential selection can stabilize the coexistence of cooperation and defection. For ε = 1.0, the cooperative clusters can even outperform the defectors, and cooperation becomes the dominant strategy in the population even if the temptation b is not small. In addition, we studied the distribution of cooperative neighbors of the focal cooperators and focal defectors in Fig 5. As shown in Fig 5(a), for ε = 0.0, cooperative neighbors can hardly survive around the focal defectors. This is because defection is more advantageous, and cooperators can only form compact clusters to resist the invasion of defectors. For ε = 0.5, about half of the focal cooperators are completely surrounded by cooperative neighbors, indicating that the introduction of preferential selection can facilitate the formation of compact cooperative clusters. For ε = 1.0, the proportion of focal cooperators with four cooperative neighbors is even more than 0.8, again confirming our conclusion.
It is significant to explore the influence of noise K on the level of cooperation. The above results are obtained when K = 0.1. Fig 6 shows that qualitative features of the results remains unchanged under different noise K, that is, the proportion of cooperators increases with the strength of preferential selection. In addition, for ε = 0.0, we can observe that there exist an optimal K value that enables the cooperator to survive, which is consistent with previous work. For ε > 0.0, the increase of noise K is conducive to promoting cooperation.
Finally, we have also explored the robustness of the above results for other kinds of topological structures. Fig 7 shows contour plots encoding the cooperation level F C on the ε − b parameter plane, and the simulation results in Fig 7(a) and 7(b) are carried out on random regular networks (RRNs) [51] and Erdös-Rényi (ER) random [52] networks with the same average degree hki = 4. From Fig 7, we can observe that the steady cooperation level F C   Fig 4. Characteristic snapshots of the distribution of different individuals on square lattices with a prepared initial condition. Red and yellow represent cooperators and blue and green represent defectors. More specifically, yellow denotes a cooperator whose strategy in the last round was defection, while green marks a defector whose strategy in the last round was cooperation. The values of ε are set to be 0 (top row), 0.5 (middle row), 1 (bottom row) and t = 1, t = 10, t = 50, t = 100, t = 10000 from left to right. The game parameter b is fixed to 1.1. https://doi.org/10.1371/journal.pone.0282258.g004

PLOS ONE
Towards preferential selection in the prisoner's dilemma game

PLOS ONE
Towards preferential selection in the prisoner's dilemma game increases with the strength of the preferential selection ε, regardless of whether the underlying interaction network is an RRN or ER network, and the introduction of the preferential selection mechanism can enhance the emergence of cooperation in social dilemmas. This is also consistent with the numerical results shown in Fig 1(b), which are carried out on square lattices, except that higher equilibrium F C could be obtained on RRN and ER networks due to stronger network reciprocity. These results indicate that the preferential selection mechanism facilitates the evolution of cooperation significantly regardless of the underlying topological structure.

Conclusion
In this paper, we have investigated the effects of the selection scheme of role models on cooperation in spatial evolutionary PDGs. We have proposed a weak PDG model by combining pure random selection and preferential selection rules. Specifically, we have introduced a normalized parameter ε to control the weights of these two types of selection rules. We have focused on the transition towards preferential selection in the spatial PDG by controlling the tunable parameter ε to guide the system from pure random selection to preferential selection. Through numerical simulations, we found that the introduction of a preferential selection mechanism can significantly alleviate social dilemmas and enhance the effect of network reciprocity, which can effectively enhance the emergence and maintenance of cooperative behaviors among selfish individuals. The critical threshold of the temptation b for the annihilation of cooperative behaviors becomes larger as ε increases, which means that the stronger the preferential selection, the more conducive it is to the evolution of cooperation. Moreover, we have provided some detailed explanations for the above simulation results from the perspective of the fraction of strategy transition and the nature of the cooperative clusters. Finally, we have discussed the robustness of the results for noise K and different topologies. We believe our study takes the understanding of preferential selection in spatial evolutionary games one step further.